On the Bias and Stability of the Results of Comparative Judgment

Comparative judgment is a method that allows measurement of competence by comparison items with other items. In educational measurement, where comparative becoming an increasingly popular assessment method, are mostly students’ responses to assignment or examination. For assessments using judgment, the Scale Separation Reliability (SSR) used estimate reliability measurement. Previous research has shown SSR may overestimate when pairs be compared selected certain adaptive algorithms, raters use different underlying models/truths, true variance item parameters below one. This investigated bias and stability components in relation number comparisons per increase understanding SSR. We showed many required obtain accurate variance, but can useful even overestimated. Lastly, we recommend adjusting general guideline for 41 item. recommendation partly depends on our simulation study needs further investigation.